Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bit rate form. Compression can be lossless, in which case quality of the video when reconstructed does not suffer but decreases in bit rate are limited by the complexity of the video. Or, compression can be lossy, in which case quality of the reconstructed video suffers but decreases in bit rate are more dramatic. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.
Over the last two decades, various video codec standards have been adopted, including the ITU-T H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263 and H.264 (MPEG-4 AVC or ISO/IEC 14496-10) standards, the MPEG-1 (ISO/IEC 11172-2) and MPEG-4 Visual (ISO/IEC 14496-2) standards, and the SMPTE 421M (VC-1) standard. More recently, the ITU-T H.265 standard (H.265, HEVC, or ISO/IEC 23008-2) has been approved. Various extensions to the H.265 standard are under development. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard also provides details about decoding operations a decoder should perform to achieve conforming results in decoding. Aside from codec standards, various proprietary codec formats define other options for the syntax of an encoded video bitstream and corresponding decoding operations.
In general, for lossy compression, a video encoder trades off bit rate against distortion introduced during the compression process. Distortion is introduced to simplify video content, thereby enabling more dramatic reductions in bit rate. The encoder can use more bits to compress video content while introducing less distortion, so that the reconstructed version of the video has higher quality. Or, the encoder can use fewer bits to compress the video content while introducing more distortion, so that the reconstructed version of the video has lower quality. In practice, the number of bits available to compress video is usually limited by network bandwidth or storage considerations. Thus, the goal of the encoder becomes providing the best overall quality for reconstructed video, subject to bit rate limits (and possibly other constraints such as delay constraints). For example, the encoder can allocate more bits to complex regions of video (which are less compressible, and hence need more bits to attain a given quality level) and allocate fewer bits to simpler regions (which need fewer bits to attain the given quality level). In many video delivery scenarios, a video encoder attempts to provide video at a constant bit rate (“CBR”). For CBR encoding, the encoder typically uses an output buffer to produce encoded data at a constant, target bit rate. In addition to providing feedback to the encoder to control quality and bit rate, the output buffer provides limited room to “smooth out” fluctuations above/below the target bit rate, where such fluctuations are due to changes in the intrinsic complexity of the video. On the other hand, for variable bit rate (“VBR”) encoding, the encoder attempts to provide reconstructed video with constant or relatively constant quality, even if bit rate varies significantly. For example, the encoder uses a smoothly variable quantization parameter (“QP”) for different types of frames, such that quality is uniform or almost uniform, but output bit rate may vary depending on the content of the video.
An encoder uses “rate control” to manage bit rate and quality during encoding. For most standards and formats, the encoder sets a QP for a picture or portion of a picture. The value of QP affects bit rate and quality. As the encoder increases the value of QP, bit rate decreases but the quality of the reconstructed video also tends to decrease. As the encoder decreases the value of QP, bit rate increases and the quality of the reconstructed video also tends to increase. Many previous approaches to rate control fall short of optimal quality for a given bit rate, fail to consistently produce encoded video at a target bit rate, or fail to adapt quickly to scene changes in video. This is especially common when a video encoder adapted to encode camera video instead encodes screen capture content, e.g., for remote desktop conferencing. Other previous approaches to rate control, such as previous approaches to so-called rho-domain rate control, regulate quality and bit rate more effectively but are too computationally intensive to use in many scenarios.